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REMARKS 

In the Office Action, the Examiner rejected Claims 1-22, which were all of the men 
pending claims, under 35 U.S.C. §103 as being unpatentable over U.S. Patent 5,767,842 (Korth) 
in view of U.S. Patent 6,407,679 (Evans, et al.). 

Independent Claims 1, 4, 8, 9, 10, 14, 17 and 21 are being amended to better define the 
subject matters of these claims, and Claim 7 is being amended to be dependent from Claim and 
to better describe the correlator module described in the claim. Claims 13 and 20 are being 
cancelled to reduce the number of issues in this case. New Claims 23 and 24, which are 
dependent from Claim 10, are being added to describe preferred features of the invention. 

For the reasons set forth below, Claims 1-12, 14-1 9 and 21-25 patentably distinguish over 
the prior art and are allowable. The Examiner is, thus, respectfully asked to reconsider and to 
withdraw the rejection of Claims 1-12, 14-19, 21 and 22, and to allow these claims and new 
Claims 23-25. 

The present invention relates to methods and systems for generating text or data from 
typing gestures made without any real keyboard. To do this, a person moves his or her hands as 
if that person was typing, and various computer processes are transform those finger movements 
or gesture into text or data. 

For example, with the preferred embodiment of the invention, images of the finger 
gestures are taken. The computer processes are then used to classify these gesture images into 
classes, to associate each of those classes with one or more possible keys, and to assign a 
probability to each of those possible keys. After a sequence of classes, the probabilities assigned 
to the possible keys are integrated to identify a word for that sequence of classes. 
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Korth describes a virtual typing system, and in this system, hand and finger movements 
are interpreted as operations on a physically non-existent computer keyboard or similar input 
device. 

Evans, et al. describes a method and system in which finger movements are converted 
into text 

There are several important differences between the preferred embodiment of this 
invention and the prior art(Korth). 

Korth assumes that the user sees a virtual keyboard (that is either printed on a table) or is 
seen on a monitor. In the present invention preferably, in the main case, the user does not see a 
virtual keyboard at all. The user may even not have a table in front of himself. Therefore in this 
invention, the typing processes are really a set of gestures that imitate some typical typing 
movements but they are so vague that they require a statistical recognition process that interprets 
gestures, like speech recognition machine interprets speech into words. Therefore, the present 
invention uses a notion of USER DEPENDENT virtual keyboard (see p. 14 of the patent 
application). The virtual keyboard can be completely imaginary (without any real physical 
representation either as an image on a table or on a display). 

Also, since different users have different ways to represent via gestures their intention 
(what key they assume with those or other finger movements) the system of this invention needs 
to learn how their gestures correspond to intentions. This is analogous to how a speech 
recognition system is trained. There is a great variability in representation sounds by different 
people. Therefore, the speech recognition system is trained to understand how different people 
pronounce the same words and sounds. And since there is such a large variability in how people 
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produce sounds, a typical method in speech recognition is to use a statistical processor that 
involves an acoustic statistical processor and language model filter to map sounds into words. 

A significant, non-obvious aspect of the present invention is to represent recognition of 
gestures via a statistical machine that has two components - one is gesture features statistical 
component, and the other is a language model filter, a special case of a statistical gesture 
machine that is considered in the present patent application is an HMM (see figure 7). 

The preferred gesture class of this invention is analogous to phonemes, like "A", "B", n C" 
exists in a phonetic system. The structure for gesture classes comes from typical movements that 
users do when they type on a keyboard. Therefore, Applicants respectfully submit, the 
Examiner's critic of the gesture classes of this invention as something that is similar to a prior art 
not correct This is a fundamental new concept that allows a statistical machine like HMM to 
work. 

Continuing the comparison with speech recognition statistical methods: one important 
feature of speech HMM - it recognizes a word not by a sequence of guessed sounds, but as an 
interpretation of a whole sequence of some classes of sounds. For example, if someone speaks 
the word "Table". The HMM based speech recognition system does not recognize a word Table 
as a sequence of sounds "T", "a", "b", "1" , V\ Instead, it has a special acoustic HMM model for 
a word "Table" and recognizes the word Table by considering a sequence of some classes of 
sounds using some hidden processes and matching possible sequences of sounds to different 
word models in the vocabulary. Something similarly and highly not-trivial is done by 1he present 
invention for gesture recognition. 
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More specifically, the preferred embodiment of this invention provides a totally 
keyboard- free INVISIBLE gesture-key mapping system free from representation of a keyboard 
in any form (even as printed on a table or displayed on a monitor) that is based on a statistical 
machine that interprets sequences of typing like gestures classes as intended words based on user 
typing gesture models. These user gesture models are trained - either for some specific user 
(and become user dependent) or it can be trained on a large class of users and become user 
independent. 

In order to emphasize this aspect of the invention, the independent Claims 1, 4, 8, 9, 10, 
14, 17 and 21 are being amended to describe the virtual keyboard, or the keyboard relative to 
which the gestures are made, as being invisible. This distances these claims from the prior art 
that uses a virtual but visible in some form keyboard. In particular, Claim 1 is being amended to 
indicate that the virtual keyboard is an invisible keyboard, and to indicate that each of the gesture 
classes is associated with one or more possible keys of the invisible keyboard. Claims 4, 8, 9, 
10, 14 and 17 are being amended to describe the feature that the captured or typing gestures are 
relative to an invisible keyboard, and that the gesture classes are associated with one or more 
possible keys of that invisible keyboard. 

Another important feature of the preferred embodiment of this invention is shown in Fig. 
7, and described in new Claim 24, where gesture input is represented as waveforms that are 
digitized and clustered in gesture frames and then processed by HMM machines that represent 
different models for different typing patterns. 
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Applicants would like to note that a further, significant advancement of the preferred 
embodiment of this invention is that, as described in Claim 7, the position correlator (described 
on p. 13 of the application) allows the INVISIBLE keyboard to be positioned correctly in case 
the user hand position changes, as opposed to the visible virtual keyboard in the prior art 

With respect to Evans, et al, it is noted that there is a very important, general difference 
between the procedure of mis invention and the system disclosed in Evans, et al. This difference 
is that with the procedure of Evans, et al, the words are determined on a sentence-by-sentence 
base; while with the present invention, the words are determined, as explained above, on a word- 
by-word basis. Because of this, Evans, et al, does not disclose or suggest, among other matters, 
the specific gesture-to-key conversion process of the present invention. 

In particular, Evans, et al, does not suggest the feature, described in Claims 1, 8, 9, 10 
and 17 that the compute processes described in these claims associate each of the classes with 
one or more possible keys, assign a probability to each of those possible keys, and integrates 
those probabilities assigned to the possible keys to identify a word for a sequence of gestures, 
likewise, Evans, et al dose not suggest the limitation of Claim 4 that the associator module 
associates gesture classes with one or more possible keys, and that the integrator module 
integrates those probabilities assigned to the possible keys to identify a word for a sequence of 
gestures. Claim 14 describes the feature, that is not shown or suggested in Evans, et al, of a 
producing step including the steps of associating each of the classes with one or more possible 
keys, assigning a probability to each of said possible keys, and integrating those probabilities 
assigned to the possible keys to identify a word for a sequence of gestures. Claim 21 is directed 
to a typing system and sets forth analogous apparatus limitations. 
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Another important difference between Evans and the preferred embodiment of the 
present invention relates to the statistical process used in this invention, of input data to get a 
text. The preferred statistical process of this invention has probability distribution on back input 
(gestures) and output (words or keys). Evan appears to have probability distribution only on 
letters and words (what sequence of words and letters is most probable). 

Therefore, Evans basically has only what corresponds to language component in the 
preferred statistical machine of this invention. But Evans does not have another statistical 
gesture component (that requires a probability distribution for gestures and keys). 

The other references of record have been reviewed, and these other references, whether 
considered individually or in combination, also do not disclose or suggest the subject matters of 

Claims 1, 4, 8, 9, 10, 14, 17 and 21. 

Because of the above-discussed differences between Claims 1, 4, 8, 9, 10, 14, 17 and 21 
and the prior art, and because of the advantages associated with those differences, these claims 
patentably distinguish over the prior art and are allowable. Claims 2 and 3 are dependent from 
Claim 1 and are allowable therewith; Claims 5, 6 and 7 are dependent from, and are allowable 
with, Claim 4; and Claims 11, 12, 22 and 23 are dependent from Claim 10 and are allowable 
therewith. Similarly, Claims IS and 16 are dependent from, and are allowable with, Claim 14; 
Claims 18 and 19 are dependent from Claim 17 and are allowable therewith; and Claim 22 is 
dependent from, and is allowable with, Claim 21. The Examiner is, accordingly, respectfully 
requested to reconsider and to withdraw the rejection of Claims 1-12, 14-19, 21 and 22, and to 
allow these claims and new Claims 23 and 24. 
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Every effort has been made to place this case in condition for allowance, a notice of 
which is respectfully requested. If the Examiner believes that a telephone conference with 
Applicants* Attorneys would be advantageous to the disposition of this case, the Examiner is 
asked to telephone the undersigned. 

Respectfully submitted, 

/Tohn S. Sensny a 
' Registration No. 28,757 
Attorney for Applicants 

SCULLY, SCOTT, MURPHY & PRESSER 
400 Garden City Plaza 
Garden City, New York 11530 
(516)742-4343 

JSS:jy 
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